System and methods for identifying vessels within tissue

ABSTRACT

A method includes providing a graphical representation of a surgical site, grasping the tissue of the patient with first and second jaw members of an end effector including a location sensor, illuminating optical light into the grasped tissue at the first jaw member, receiving the optical light that has passed through the grasped tissue, at the second jaw member, processing the received light to identify a vessel, which is encompassed within the grasped tissue, receiving location information of the end effector from the location sensor, synchronizing a location of the identified vessel within the graphical representation based on the location information, and displaying the identified vessel at the synchronized location in the graphical representation.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of and priority to U.S. Provisionalpatent Application Nos. 62/967,238; 62/967,241; and 62/967,246, filed onJan. 29, 2020, the entire contents of each of which is herebyincorporated herein by reference.

FIELD

The present disclosure is generally related to systems and methods foridentifying a location of vessel within tissue with a light source,displaying the identified vessel at a proper location, and/or confirmingsealing of the identified vessel.

BACKGROUND

Surgical operations including laparoscopic operations involve operationson tissue. During an operation, tissue can be cut, coagulated, and/orsealed. Complications may occur when the tissue includes vessels, suchas blood, vile, or lymph vessels. For example, when tissue is planned tobe cut and the tissue includes a blood vessel, the blood vessel mightalso be also cut and leak blood or body fluid around the tissue, therebycausing complications. Thus, identification of vessels, which are notseen from outside or are hidden within the tissue, is advantageous.

During laparoscopic operations, a surgeon has a limited view. Thus,display of hidden vessels at an appropriate place on a display is alsoadvantageous in increasing certainty of efficacy of the surgicaloperations and decreasing potential harm to patients.

Further, prior to or concurrently with surgical operations, completionof sealing of vessel needs to be confirmed so as to also increasecertainty of efficacy of the surgical operations and decrease potentialharm to patients.

SUMMARY

This disclosure generally relates to identification of vessels hiddenwithin tissue, display of the identified vessels at the proper location,and/or confirmation of sealing of vessels so that performance andefficacy of surgical operations can be increased.

Provided in accordance with aspects of the disclosure is a method foridentifying a vessel within tissue. The method includes providing agraphical representation of a surgical site, grasping the tissue of thepatient with first and second jaw members of an end effector including alocation sensor, illuminating optical light into the grasped tissue atthe first jaw member, receiving the optical light that has passedthrough the grasped tissue, at the second jaw member, processing thereceived light to identify a vessel, which is encompassed within thegrasped tissue, receiving location information of the end effector fromthe location sensor, synchronizing a location of the identified vesselwithin the graphical representation based on the location information,and displaying the identified vessel at the synchronized location in thegraphical representation.

In an aspect of the disclosure, the method further includes supplyingenergy to the tissue through the end effector to seal the identifiedvessel, illuminating the optical light into the identified vessel aftersupplying the energy, and receiving the optical light that has passedthrough the identified vessel.

In an aspect of the disclosure, the method further includes processingthe received optical light after supplying the energy.

In another aspect of the disclosure, the method further includesconfirming whether or not the identified vessel is sealed. Theidentified vessel is confirmed to be sealed by comparing the identifiedvessel before supplying the energy with the identified vessel aftersupplying the energy.

In still another aspect of the disclosure, the graphical representationis a 3D model or a video image.

In still another aspect of the disclosure, the vessel is selected fromthe group consisting of a bile vessel, a lymph vessel, and a bloodvessel.

In yet another aspect of the disclosure, the method further includesgenerating an electromagnetic wave. In another aspect, the methodfurther includes sensing the electromagnetic wave. The locationinformation is based on the sensed electromagnetic wave.

In yet still another aspect of the disclosure, the identified vessel isdisplayed in an augmented manner in the graphical representation.

Provided in accordance with aspects of the disclosure is a system foridentifying a vessel within tissue. The system includes an end effectorconfigured to grasp tissue, the end effector including a first jawmember and a second jaw member, an optical light source configured toemit optical light to the grasped tissue and fixed at the first jawmember, a light sensor configured to receive the optical light that haspassed through the grasped tissue, and fixed at the second jaw member, alocation sensor configured to detect location information of the endeffector, a processor configured to receive a graphical representationof a surgical site, process the received light to identify a vessel,which is encompassed within the grasped tissue, and synchronize alocation of the identified vessel within the graphical representationbased on the location information, and a display configured to displaythe identified vessel at the synchronized location in the graphicalrepresentation.

In an aspect of the disclosure, the system further includes an energysupplier configured to supply energy through the end effector to sealthe identified vessel. The optical light source is further configured toilluminate the optical light into the identified vessel after supplyingthe energy, and the location sensor is further configured to receive theoptical light that has passed through the identified tissue.

In another aspect of the disclosure, the processor is further configuredto process the received light after supplying the energy.

In another aspect of the disclosure, the processor is further configuredto confirm whether or not the identified vessel is sealed.

In another aspect of the disclosure, the processor confirms that theidentified vessel is sealed by comparing the identified vessel beforesupplying the energy with the identified vessel after supplying theenergy. The graphical representation is a three-dimensional model or avideo image.

In another aspect of the disclosure, the vessel is selected from thegroup consisting of a bile vessel, a lymph vessel, and a blood vessel.

In still another aspect of the disclosure, the system further includesan electromagnetic wave generator configured to generate anelectromagnetic wave.

In yet another aspect of the disclosure, the system further includes anelectromagnetic sensor configured to sense the electromagnetic wave. Thelocation information is based on the sensed electromagnetic wave.

In yet still another aspect of the disclosure, the identified vessel isdisplayed in an augmented manner in the graphical representation.

The details of one or more aspects of the disclosure are set forth inthe accompanying drawings and the description below. Other features,objects, and advantages of the techniques described in this disclosurewill be apparent from the description and drawings, and from the claims.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram of a surgical system for identifying vesselsaccording to aspects of the present disclosure;

FIG. 2 is a perspective view of an energy-delivery device including anend effector assembly in accordance with aspects of the presentdisclosure;

FIGS. 3A and 3B are cross-sectional side views of an end effectorassembly of an energy delivery device having jaw members for graspingtissue and blood vessels according to aspects of the present disclosure;

FIG. 4 is a cross-sectional front view of the energy delivery device ofFIGS. 3A and 3B for identifying vessels in tissue with an optical lightsource according to aspects of the present disclosure;

FIG. 5 is a cross-sectional front view of an end effector assembly of anenergy delivery device having jaw members for grasping tissue and bloodvessels and identifying vessels in tissue with a laser source accordingto embodiments of the present disclosure;

FIG. 6 depicts raw laser speckle data in accordance with aspects of thepresent disclosure;

FIG. 7 is a laser speckle contrast image (LSCI) illustrating subsurfaceblood flows in tissue obtained based on laser speckle data prior tovessel sealing according to aspects of the present disclosure;

FIG. 8 is an LSCI illustrating subsurface blood flows of the sameportion of the tissue as FIG. 7 after vessel sealing;

FIG. 9 is a cross-sectional front view of an end effector assembly of anenergy delivery device having jaw members for grasping tissue and bloodvessels and identifying vessels in tissue with a laser source based onRaman Shift according to aspects of the present disclosure;

FIGS. 10A-10C are graphical representations of Raman Shifts according toaspects of the present disclosure;

FIG. 11 is a flowchart illustrating a method for identifying bloodvessels based on optical light according to aspects of the presentdisclosure;

FIG. 12 is a flowchart illustrating a method for identifying bloodvessels based on Raman shifts according to aspects of the presentdisclosure;

FIG. 13 is a flowchart illustrating a method for identifying bloodvessels based on laser speckle data according to aspects of the presentdisclosure; and

FIG. 14 is a block diagram for a computing device according to aspectsof the present disclosure.

DETAILED DESCRIPTION

Surgical operations often involve operations to tissue that includesvessels. When the tissue is to be cut or treated, the vessels in thetissue should be sealed so as to prevent blood or body fluid fromleaking. Thus, identification of the vessels in the tissue enhancesperformance of the surgical operations. Further, prior to, concurrentlywith, or after the treatment of the tissue, completeness of sealingvessels should be confirmed. Thus, the present disclosure providessystems and methods for identifying vessels, which are hidden withintissue, displaying the identified vessel at an appropriate location,and/or confirming completeness of the vessel seal.

Different types of light sources and light sensors may be used toidentify a location of a vessel. For example, an optical light source orlaser light source is employed in this disclosure, although other lightsources are also contemplated. Identification of vessels may be based onoptical light, reflected laser spectrum, Raman shift, or in any othersuitable manner. Corresponding structures and methods are described inthe following description and in the drawings.

FIG. 1 shows a block diagram of a surgical system 100 for identifyingvessels according to embodiments of the present disclosure. The surgicalsystem 100 may use any type of energy to seal tissue includingmechanical energy, acoustic energy, thermal energy, electrical energy,or electromagnetic (EM) energy (e.g., optical energy or radio frequency(RF) energy). The surgical system 100 may use EM waves to identify alocation of one or more elements of the surgical system 100 andsynchronize the patient with a three-dimensional (3D) model of apatient. Ultrasound may be used by the surgical system 100 to identify alocation of elements of the surgical system 100. Further, the surgicalsystem 100 may identify a location of vessels hidden in tissue ofinterest, synchronize the identified location of the vessel with the 3Dmodel, and display a graphical representation of the vessel at thecorresponding location in the 3D model in an augmented way. By doing theabove, the surgical system 100 helps clinicians to perform surgicaloperations without unintentionally cutting or otherwise damagingvessels, e.g., blood vessels.

In embodiments, the surgical system 100 may use EM waves to identify alocation of one or more elements of the surgical system 100 andsynchronize the patient with a live video of a patient. Further, thesurgical system 100 may identify a location of vessels hidden in tissueof interest, synchronize the identified location of the vessel with thelive video, and display a graphical representation of the vessel at thecorresponding location in the live video in an augmented way. By doingthe above, the surgical system 100 helps clinicians to perform surgicaloperations without unintentionally cutting or otherwise damagingvessels, e.g., blood vessels.

Prior to or concurrently with surgical operations, a three-dimensional(3D) model is generated to visually display patient's anatomy. During animaging/planning stage, a computer utilizes computed tomography (CT)image data or other image data in the Digital Imaging and Communicationsin Medicine (DICOM) format or similar format, for generating and viewinga 3D model of the patient's body. In embodiments, the 3D model may begenerated in real time based on the live video. The 3D model and imagedata derived from the 3D model enables identification of the region ofinterest (automatically, semi-automatically or manually), and allows forthe selection of a pathway to the region of interest. More specifically,the CT scans are processed and assembled into a 3D volume, which is thenutilized to generate the 3D model of the patient's body. The surgicalsystem 100 may include a memory 175 to store the 3D model or receive the3D model from another computer, which has generated or stored the 3Dmodel. The surgical system 100 may be coupled to a display 170 and causethe display 170 to display the 3D model on its screen.

The surgical system 100 may include a power supply 110, an energy outputstage 120, and an instrument 130. The power supply 110 supplies power tothe energy output stage 120, which generates energy and provides theenergy to the instrument 130. The instrument 130, in turn, applies thegenerated energy to the tissue 190, which includes at least one vessel.For an RF-based tissue-sealing system, the energy output stage 120generates RF energy and the instrument 130 applies the RF energy to thetissue 190 through at least one contact to seal the tissue 190. Variousother types of instruments 130 may be encompassed in this disclosure asunderstood by a person having ordinary skill in the art.

The surgical system 100 may also include a sensor 140, a processor 160,a user interface 165, and display 170. The sensor 140 senses variousparameters and/or properties of the RF energy applied by the instrument130 at the operating site and transmits sensor signals representing thesensed parameters or properties of the RF energy to the processor 160.The processor 160 processes the sensor signals and generates controlsignals based on the processed sensor signals to control the powersupply 110 and/or the energy output stage 120. For example, theprocessor 160 may regulate the voltage or current output from the powersupply 110 or the energy output stage 120 based on the processed sensorsignals.

The sensor 140 is configured to measure various electrical orelectromechanical conditions at the operating site such as tissueimpedance, changes in tissue impedance, tissue temperature, changes intissue temperature, leakage current, applied voltage, and appliedcurrent. The sensor 140 may sample, continuously measure, and adjust oneor more of these conditions so that the processor 160 can continuallyadjust the energy output from the power supply 110 and/or the energyoutput stage 120 during a sealing procedure. For example, in an RF-basedvessel sealing, the sensor 140 may measure tissue impedance and theprocessor 160 may adjust the voltage generated by the energy outputstage 120.

The user interface 165 is coupled to the processor 160 allowing a userto control various parameters of the energy applied to the tissue 190during a surgical procedure. For example, the user interface 165 mayallow a user to manually set, regulate and/or control one or moreparameters of the energy delivered to the tissue 190, such as voltage,current, power, frequency, and/or algorithm control parameters, e.g.,pulse width, duty cycle, crest factor, and/or repetition rate.

The processor 160 may be designed to execute software instructions,which are saved in the memory 175, for processing data received from theuser interface 165 and for outputting control signals to the powersupply 110 and/or the energy output stage 120. The software instructionsmay be uploaded to or stored in an internal memory of the processor 160,an internal or external memory bank accessible by the processor 160and/or an external memory, e.g., an external hard drive, floppydiskette, or CD-ROM. Control signals generated by the processor 160 maybe converted to analog signals by a digital-to-analog converter (DAC)(not shown) before being applied to the power supply 110 and/or energyoutput stage 120.

For embodiments of an RF-based tissue-sealing system, the power supply110 is a high-voltage DC power supply that produces RF current. In theseembodiments, the processor 160 transmits control signals to the powersupply to control the magnitudes of the RF voltage and current outputfrom the power supply 110. The energy output stage 120 receives the RFcurrent and generates one or more algorithm control signals of RFenergy. The processor 160 generates algorithm control signals toregulate the parameters of the RF energy, such as pulse width, dutycycle, crest factor, and repetition rate. In other embodiments, thepower supply 110 is an AC power supply, and the energy output stage 120may vary the waveform of the AC signal generated by the power supply 110to achieve a desired waveform.

As described above, the surgical system 100 includes the user interface165, which includes an input device, such as a keyboard or touch screen,through which a user enters data and commands. The data may include thetype of instrument, the type of procedure, and/or the type of tissue.The commands may include target effective voltage, current, or powerlevel, or other commands for controlling parameters of the energy thatis delivered from the energy output stage 120 to the instrument 130.

In embodiments, the user interface 165 may be incorporated into thedisplay 170. For example, the display 170 may be touch sensitive anddisplay graphical icons/representations to adjust various parameters. Insuch configurations, a clinician adjusts values of the variousparameters by touching/holding/dragging icons on the display 170.

The surgical system 100 may also include a light source, e.g., anoptical light source 150, and a light sensor 155. The optical lightsource 150 emits optical light to the tissue 190 and the light sensor155 senses light, which has passed through or reflected from the tissue190. In embodiments, the light source 150 may be coupled to the energyoutput stage 120. The energy output stage 120 may generate light thatmay be the same as the electrosurgical energy applied to the tissue 190to perform an electrosurgical procedure (e.g., vessel sealing).Alternatively, the energy output stage 120 may generate light that hasparameters that are different from the parameters of the electrosurgicalenergy applied to the tissue 190.

The light sensor 155 generates a sensor signal or sensor data based onthe sensed light and transmits the sensor signal or sensor data to theprocessor 160, which processes the sensor signal or sensor data todetermine the level of blood circulation in the tissue 190 and toidentify a location of vessels in the tissue 190. For example, theprocessor 160 may determine the level of blood circulation and alocation of a vessel based on the magnitude, phase, scatter, or Dopplereffect of the sensor signal or the sensed light.

The sensed light may also provide information about the tissue type. Forexample, the sensed light may identify the tissue as connective tissue,muscle tissue, nervous tissue, vascular tissue, epithelial tissue, orany other tissue type or combination of tissue types. The sensed lightmay also identify the vessel type within the tissue 190. The vesseltypes include bile vessels, lymph vessels, and blood vessels. The sensedlight may distinguish the type of blood vessel that resides in a givenportion of tissue. The types of blood vessels include arteries,arterioles, capillaries, venules, and veins. The sensed light may alsobe used to identify the condition of the tissue, such as whether thetissue is diseased and/or damaged.

The surgical system 100 may determine the level of blood circulation bysensing tissue parameters or properties that depend on the level ofblood circulation during a period exceeding one cardiac cycle. Inembodiments, the surgical system 100 may sample tissue parameters orproperties for multiple cardiac cycles to more accurately determine thelevel of blood circulation. In other embodiments, a cardiac signal,which is related to heart contractions (e.g., an electrocardiographicsignal), can be used to evaluate the correlation between the parametersof the sensor signal and the cardiac signal to more accurately assessthe level of blood circulation.

In an embodiment, the light source 150 may emit optical light or radiatelaser light. The light sensor 155 may be a charged coupled device (CCD)or complementary metal-oxide semiconductor (CMOS). The light sensor 155may capture an image of the vessels hidden in the tissue 190. Theprocessor 160 may then analyze the image and identify a location of thevessel. The processor 160 may also utilize the sensed results from thesensor 140 together with the image in identifying the location of thevessels.

In another embodiment, the light source 150 and the light sensor 155 maybe installed or incorporated into the instrument 130. For example, theinstrument 130 may be a forceps, which has two jaw members. The lightsource 150 may be installed on one of the two jaw members and the lightsensor 155 may be installed on the other of the two jaw members, asdescribed below with reference to FIGS. 2-4 and 9. In yet anotherembodiment, the light source 150 and the light sensor 155 may both beinstalled or incorporated into one of the two jaw members as describedbelow with reference to FIG. 5.

Continuing with reference to FIG. 1, when a patient is placed on asurgical table for receiving a surgical operation, an EM wave isgenerated by an EM wave generator 180. The processor 160 may control theEM wave generator 180 and perform intermittent activation of the EM wavegenerator 180. The generated EM wave surrounds the patient. An EM sensor185, which is installed/fixed on the instrument 130 a predetermineddistance from its distal tip or other point of reference, senses thestrength of the EM wave at the position of the instrument 130. Based onthe strength of the EM wave, the processor 160 is able to estimate alocation of the instrument 130 with respect to the EM coordinate system.The EM sensor 185 may be installed on another element of the surgicalsystem 100 to monitor the spatial relationship within the surgicalsystem 100. The processor 160 may synchronize the EM coordinate systemwith the coordinate system of the 3D model.

In embodiments, the EM sensor 185 may have a predetermined spatialrelationship with the light source 150 or the light sensor 155. When alocation of the vessel is identified based on the sensed results fromthe light sensor 155, the location of the vessel may be alsosynchronized with the 3D model and a graphical representation of thevessel may be displayed at the corresponding location in the 3D model inan augmented way. Thus, when the 3D model is moved or rotated, thegraphical representation of the vessel is correspondingly moved orrotated.

As an alternative or in addition to incorporating the location of thevessel into the 3D model, a graphical representation of the vessel maybe displayed at the corresponding location on a live video image of asurgical site, e.g., a video image obtained from an endoscope anddisplayed on a surgical display. The graphical representation may beoverlaid or projected onto the live video image in an augmented way. Inembodiments where video imaging is used, the location of the vessel maybe synchronized with the video image, e.g., tissue features, surgicalinstrument(s), etc. within the video image, such that when the videoimage is moved or rotated, the graphical representation of the vessel iscorrespondingly moved or rotated.

After the instrument 130 seals the vessel, the light source 150 emitslight to the grasped tissue between the jaw members and the light sensor155 senses the light, which is the light passing through or reflectedfrom the tissue 190. The processor 160 processes the sensed results fromthe light sensor 155. The vessel identified after sealing is comparedwith the vessel identified prior to sealing and/or with sealingparameters. In a case where the dimensions, optical properties, etc.(absolute or relative to the previously identified vessel) of thecurrently identified vessel indicate that the currently identifiedvessel has been sufficiently sealed, the processor 160 may confirm thatthe sealing has been performed completely. Otherwise, the processor 160determines that the sealing has not been performed completely andinforms the clinician to perform sealing again.

In embodiments, where the previously identified vessel is sealed and cutalong the seal such that the currently identified vessel includes twosealed vessel portions, the processor 160 may likewise confirmcompleteness of the sealing of the vessel portions, e.g., based upondimensions, optical properties, etc. (absolute or relative to thepreviously identified vessel).

FIG. 2 shows a forceps 200 for vessel sealing according to embodimentsof the present disclosure. The forceps 200 includes a housing 205, ahandle assembly 210, a trigger assembly 220, a rotatable assembly 230,and an end effector assembly 270. The end effector assembly 270 mayinclude any feature or combination of features of jaw members. Thecomponents of the forceps 200 are adapted to mutually cooperate tograsp, seal, divide and/or sense tissue, e.g., tubular vessels andvascular tissue. In embodiments, the trigger assembly 220 may beconfigured to actuate a cutting function, e.g., a knife or electricalcutter, of the forceps 200 or to actuate another component, as describedbelow.

The end effector assembly 270, which is described in variousconfigurations in connection with FIGS. 3A-3B, 4, 5, and 9, generallyincludes two jaw members 275 and 285 disposed in opposing relationrelative to one another. One or both of the jaw members 275 and 285 aremovable from a first position wherein the jaw members 275 and 285 aredisposed in spaced relation relative to one another to a second positionwherein the jaw members 275 and 285 cooperate to grasp tissuetherebetween.

The forceps 200 includes an elongated shaft 250 having a distal portion260 configured to mechanically engage the end effector assembly 270. Theproximal portion 255 of the shaft 250 is received within the housing205. The rotatable assembly 230 is mechanically associated with theshaft 250 such that rotational movement of rotatable assembly 230imparts similar rotational movements to the shaft 250 that, in turn,rotates the end effector assembly 270.

The handle assembly 210 includes a fixed handle 225 and a movable handle215. In embodiments, the fixed handle 225 is integrally associated withthe housing 205, and the movable handle 215 is selectively movablerelative to the fixed handle 225. The movable handle 215 of the handleassembly 210 is ultimately connected to a drive assembly (not shown). Ascan be appreciated, applying force to move the movable handle 215 towardthe fixed handle 225 pulls a drive sleeve of the drive assemblyproximally to impart movement to the jaw members 275 and 285 from thefirst position, wherein the jaw members 275 and 285 are disposed inspaced relation relative to one another, to the second position, wherethe jaw members 275 and 285 cooperate to grasp tissue locatedtherebetween.

In embodiments, the end effector assembly 270 may be configured as aunilateral assembly that includes a stationary jaw member mounted infixed relation to the shaft 250 and a pivoting jaw member movablymounted about a pin 265. The jaw members 275 and 285 may be curved atvarious angles to facilitate manipulation of tissue and/or to provideenhanced line-of-sight for accessing targeted tissues. Alternatively,the forceps 200 may include a bilateral assembly, e.g., both jaw members275 and 285 move relative to one another and shaft 250.

The forceps 200 further includes first and second switch assemblies 235and 240 configured to selectively provide energy to the end effectorassembly 270. More particularly, the first switch assembly 235 may beconfigured to perform a first type of surgical procedure (e.g., seal,cut, and/or sense) and a second switch assembly 240 may be configured toperform a second type of surgical procedure (e.g., seal, cut, and/orsense). It should be noted that the presently-disclosed embodiments mayinclude any number of suitable switch assemblies and are not limited tothe switch assemblies 235 and 240. It should further be noted that thepresently-disclosed embodiments may be configured to perform anysuitable surgical procedure and are not limited to only sealing, cuttingand sensing. Further, as noted above, cutting may be performed byactuation of the trigger assembly 220, e.g., for mechanical cutting, inaddition to or as an alternative to second switch assembly 240.

The forceps 200 may include a controller 245. In embodiments, thecontroller 245 may be provided as a separate component coupled to theforceps 200 or integrated within the forceps 200. The controller 245 mayinclude any type of computing device, computational circuit, or any typeof processor or processing circuit capable of executing a series ofinstructions that are stored in a memory. The controller 245 may beconfigured to control one or more operating parameters associated withan energy source (e.g., the power supply 110 or the energy output stage120 of FIG. 1) based on one or more signals indicative of user input,such as generated by the first and second switch assemblies 235 and 240and/or one or more separate, user-actuatable buttons or switches.Examples of switch configurations that may be suitable for use with theforceps 200 include, but are not limited to, pushbutton, toggle, rocker,tactile, snap, rotary, slide and thumbwheel. In embodiments, the forceps200 may be selectively used in either a monopolar mode or a bipolar modeby engagement of the appropriate switch.

The first and second switch assemblies 235 and 240 may also cooperatewith the controller 245, which may be configured to automaticallytrigger one of the switches to change between a first mode (e.g.,sealing mode) and a second mode (e.g., cutting mode) upon the detectionof one or more parameters or thresholds. In embodiments, the controller245 is configured to receive feedback information, including varioussensor feedback with regard to temperature of tissue, electricalimpedance of tissue, jaw closure pressure, jaw positioning, and/or othervarious feedback information, e.g., using Raman spectroscopy, laserspeckle imaging, optical imaging, fluorescence spectroscopy, and/orlaser-induced tissue fluorescence, and to control the energy sourcebased on the feedback information.

Embodiments of the present disclosure allow the jaw members 275 and 285to seal and/or cut tissue using light energy and/or RF energy. Inembodiments, the controller 245 may include a feedback loop thatindicates when a tissue seal is complete based upon one or more of thefollowing parameters: tissue temperature, light sensing, change inimpedance of the tissue over time and/or changes in the optical orelectrical power or current applied to the tissue over time, rate ofchange of these properties and combinations thereof. An audible orvisual feedback monitor may be employed to convey information to thesurgeon regarding the overall seal quality and/or the completion of aneffective tissue seal.

In embodiments, the light source 150 and the light sensor 155 of FIG. 1may be installed in or fixedly incorporated into one or both of the jawmembers 275 and 285 of FIG. 2. When the jaw members 275 and 285 movefrom the open position to the close position to grasp tissue, the lightsource 150 emits light or radiates laser to the grasped tissue and thelight sensor 155 detects the reflected or scattered light to identify alocation of the vessel in the tissue. Details of vessel identificationwill be described with respect to FIGS. 3A-13 below.

In embodiments, the surgical system 100 may be a robotic surgicalsystem, which includes one or more robotic arms. The forceps 200 may beincorporated into or fixedly installed at one robotic arm withmodifications as understood by one of ordinary skilled in the art toadapt a handheld device to one for use with a robotic surgical system.

FIGS. 3A and 3B illustrate an energy delivery device 300 including twojaw members 310 and 320, which grasp and compress tissue 330, accordingto embodiments of the present disclosure. The energy delivery device 300may be the end effector assembly 270 of FIG. 2. The jaw members 310 and320 include electrodes 305 and 315 that are electrically coupled to theenergy output stage 120 of FIG. 1. The electrodes 305 and 315 receiveenergy from the energy output stage 120 and apply it to the tissue 330and vessels 335 within the tissue 330 to perform surgical operationsonto the tissue 330 and seal the vessels 335.

As described above, the energy delivery device 300 may be also used inidentifying a location of the vessels 335 based on blood circulation ina given volume of tissue 330. To evaluate blood circulation, the givenvolume of tissue 330 is first grasped between the jaw members 310 and320 of the energy delivery device 300. The pressure that is applied tothe tissue 330 by the jaw members 310 and 320 is selected to provideelectrical contacts between the electrodes 305 and 315 and the tissue330. However, the amount of pressure applied to the tissue 330 may belower than the amount of pressure used to compress the tissue 330 duringtissue sealing. Then, a probing signal 325 (e.g., an RF signal) isapplied to the tissue 330 by the electrodes 305 and 315 and a responsesignal (e.g., tissue impedance) is measured during one or more cardiaccycles.

During the cardiac cycles, the pressure of the blood flowing in theblood vessels 335 varies and, as a result, the relative amount of bloodin a given volume of tissue 330 also varies. For example, as shown inFIG. 3A, during a first portion of the cardiac cycle, the pressure ofthe blood flowing within the blood vessels 335 is at a low level and thevolume of blood within the given volume of the tissue 330 is at a lowlevel. On the other hand, as shown in FIG. 3B, during a second portionof the cardiac cycle, the pressure of the blood flowing within the bloodvessels 335 is at a high level and the volume of blood within the givenvolume of tissue 330 is at a high level. The volume of blood within thegiven volume of tissue 330 may be measured by measuring the impedance ofthe tissue 330 based on the probing signal 325 to the tissue 330 andsensing the response signal.

During a cardiac cycle, as the volume of blood in a given volume oftissue increases, a force is applied to the jaw members 310 and 320 tourge the jaw members 310 and 320 apart from one another. In embodiments,the surgical system 100 includes a motion sensor configured to sense thechange in distance between the jaw members 310 and 320. This distanceinformation may be used together with the response signal 104 toevaluate the level of blood circulation within a given volume of tissue330.

As described above, a probing signal 325 is applied to the vessels 335and the response signal is measured over time to identify the tissue 330and/or the vessels 335 or to determine parameters of the tissue 330and/or the vessels 335. The response signal may include the frequencyand amplitude of an electrical impedance of the tissue 330. If thefrequency of the electrical impedance correlates to the frequency ofcardiac contractions, then the vessels 335 are identified as a bloodvessel. If the vessels 335 are identified as a blood vessel, theamplitude of the electrical impedance would indicate the level of bloodcirculation.

In embodiments, existence of a blood vessel in the tissue 330 may bedetermined first based on the response signal from the probing signal325. In a case when it is determined that the blood vessel exists in thetissue 330, the location of the blood vessel may then be identified, asdetailed above. Existence of a blood vessel in the tissue 330 may bealso determined by measuring a change in distance or force between thetwo jaw members 310 and 320.

FIG. 4 shows a cross-sectional front view of the energy delivery device300 of FIGS. 3A and 3B. In this embodiment, jaw members 310 and 320grasp and deform the tissue 330 by compressing the tissue 330, e.g.,extending or stretching the tissue 330 along the length-wise axis of thetissue 330, to intensify the release of elastin and collagen. The upperjaw member 310 includes a cavity 410, in which a light sensor 420 may bepositioned. The cavity 410 may be filled with a transparent material sothat light can pass through. The bottom portion of the upper jaw member310 is shaped to mate with the rounded upper portion of the lower jawmember 320, although other configurations are also contemplated such asgenerally planar jaw surfaces or other complementary jaw configurations.

The lower jaw member 320 includes an optical light source 430 and anaperture 440. The optical light source 430 emits optical light 450,which passes through the tissue 330 via the aperture 440. In an aspect,the lower jaw member 320 may also include a light distribution element(not shown) so that the optical light 450 may be uniformly transmittedto the tissue 330.

In an aspect, the light distribution element may be disposed in thelower jaw member 320 by a predetermined distance from the tissue 330.The light distribution element may include optical fibers, lenses,and/or prisms optically coupled to the light source 430 via a lightguide. The optical fibers may contain a grating structure to distributethe optical light 450 out of the side of the optical fibers along apredetermined length of the optical fibers.

As the jaw members 310 and 320 are brought together to deform the tissue330, the two sides of the upper jaw member 310 stretch or extend thetissue 330, which is to be illuminated by the optical light 450 acrossthe upper portion of the lower jaw member 320. Consequently, thedifferent layers of tissue 330 (e.g., the opposite walls of the vessel335 of FIGS. 3A and 3B) are made thinner and are brought into contactwith each other.

The propagation direction and the wavelength of the optical light 450are selected to provide the desired tissue penetration depth by theoptical light 450. Since neither the light distribution element nor thejaw members 310 and 320 have direct physical contact with the sealedvascular tissue, the sealed vascular tissue never adheres to any portionof the jaw members 310 and 320. In this manner, the jaw members 310 and320 and the light distribution element avoid contamination by the sealedtissue 330.

When the optical light 450 passes through the grasped tissue 330, theoptical light 450 is scattered or dispersed, and the dispersed orscattered light 460 may be sensed by the light sensor 420, which may bea charged coupled device (CCD) or complementary metal-oxidesemiconductor (CMOS). The light sensor 420 may capture an image of thevessels hidden in the tissue 330. The processor 160 of FIG. 1 may thenanalyze the image and identify locations of the vessels.

For example, as described above, the sensed results from the sensor 140of FIG. 1 may be used to determine whether or not one or more bloodvessels exist in the grasped tissue 330. When it is determined that ablood vessel exists within the grasped tissue 330, the optical lightsource 430 emits the optical light 450 to identify the location of theblood vessel(s) in the grasped tissue 330. The processor 160 may alsoutilize the sensed results from the sensor 140 of FIG. 1 together withthe captured image in identifying the location of the vessels. Further,after sealing the vessel, the light source 430 also emits the opticallight 450 so as to confirm whether or not the sealing has beencompleted.

FIG. 5 illustrates an energy delivery device 500 for identifying avessel(s) in tissue with a laser source according to embodiments of thepresent disclosure. The energy delivery device 500 may be of the endeffector assembly 270 of the forceps 200 of FIG. 2. Instead of having alight sensor and a light source being fixedly installed on different jawmembers as in FIG. 4, a laser source 530 and a light sensor 540 are bothfixedly installed in one of two jaw members 510 and 520. As illustrated,the laser source 530 and the light sensor 540 are installed in the lowerjaw member 520 but can be in the upper jaw member 510. Alternatively,the light sensor 540 and/or the laser source 530 may be installed in arobotic arm. The laser source 530 irradiates a laser 560, e.g., having asingle frequency, into the grasped tissue 535, and the light sensor 540detects scattered light 565, which has been reflected and scattered fromthe tissue 535.

The light sensor 540 may be an image sensor such as CCD or CMOS arrayand may output the sensed results, which may be laser speckle data forprocessing into laser speckle contrast images (LSCIs). The LSCI allows aclinician to see a quantitative mapping of local blood flow dynamics ina wide area so that the clinician can quickly and accurately assess theblood flows within the tissue 330. Thus, clinicians can see which partof the internal organs have blood vessels supplying blood so that theclinicians may be able to identify a location of blood vessels bylooking at the LSCIs in real-time.

FIG. 6 depicts a representation of raw laser speckle data 600 detectedfrom the tissue using a laser, e.g., from laser source 530, which mayhave one frequency. The light reflected and scattered from the tissuemay have different phases and amplitudes, which add together to give apattern in which its amplitude and intensity vary randomly. Generally,the raw laser speckle pattern has a Gaussian distribution pattern ofintensity. Thus, the raw laser speckle data 600 is not readily readableby a clinician and it would be exceedingly difficult to analyze the rawlaser speckle data 600 in real time and identify objects in the rawlaser speckle data 600. For example, a box 610 is located at a positionin the raw laser speckle data 600 where a blood vessel is located. Asshown, it is difficult to identify a blood vessel at the box 610 in theraw laser speckle data 600.

When there is a moving object in the area irradiated by the laser source530, the intensity fluctuates according to the movement of the object(e.g., circulating red blood cells) and thus forms a pattern differentfrom the Gaussian distribution pattern. The laser speckle contrastimaging techniques uses the speckle patterns that are fluctuated by themoving objects or the interference of many waves having the samefrequency. By analyzing the intensity fluctuation of these laser specklepatterns together with time, velocity of the moving object can beidentified. As a result, the raw laser speckle data 600 of FIG. 6 can beconverted to an LSCI of FIGS. 7 and 8 utilizing the following techniquesto render an image that is useful to clinicians in practice.

The statistics of noise-like raw laser speckle data 600 is related tospeckle contrast K containing a time component. Specifically, thespeckle contrast K includes three variables x, y, and t, where x, y, andt represent horizontal, vertical, and temporal position in the samplingspace of the laser light. The speckle contrast K(x, y, t) may be definedby a ratio of the standard deviation σ to the mean intensity I asfollows:

${{K\left( {x,y,t} \right)} = \frac{\sigma\left( {x,y,t} \right)}{{AVG}\left( {I\left( {x,y,t} \right)} \right)}},$

where σ(x, y, t) is the standard deviation of intensity in spatial andtime domain, I(x, y, t) is the intensity values of a set of pixelsadjacent to position (x, y, t) in spatial and time domain, andAVG(I(x,y,t)) is the mean or average intensity of the set of pixelsadjacent to the position (x, y, t). In embodiments, the set of pixelsmay be defined by a time series of intensity of an individual pixel,pixels in a rectangular window in the (x, y) plane at time t, or aconsecutive cubic in the (x, y, t) space.

The depth of modulation of the speckle intensity fluctuations generallygives some indication of how much of the laser light is being scatteredfrom moving objects and how much from stationary objects. Further, thefrequency spectrum of the fluctuations depends on velocity distributionof the movements of the moving objects. It follows that the specklecontrast K is related to velocity of moving objects or simply subsurfaceblood flows here. The speckle contrast K is then expressed as thefollowing equation:

${K = \left\lbrack {\frac{\tau_{c}}{2T}\left( {1 - e^{(\frac{{- 2}T}{\tau_{c}})}} \right)} \right\rbrack^{\frac{1}{2}}},$

where T is an integration time and τ_(c) is a correlation time. VelocityVis a reciprocal of the correlation time τ_(c). Thus, the specklecontrast K becomes:

${K = \left\lbrack {\frac{1}{2TV}\left( {1 - e^{{- 2}{TV}}} \right)} \right\rbrack^{\frac{1}{2}}}.$

According to this equation, when the velocity V increases, theexponential term e^(−2TV) is going to be closer to zero and the specklecontrast K is going to increase to a value which is less than

$\sqrt{\frac{1}{2{TV}}}.$

Since the velocity V is assumed to be greater than or equal to zero, thespeckle contrast K is greater than or equal to zero, and bound by

$\sqrt{\frac{1}{2TV}}.$

Based on the equation of the speckle contrast K and the velocity, thesquared value of the speckle contrast K is inversely proportional to thevelocity V, when assuming that the exponential term e^(−2TV) iscomparatively small. Or, in other words, the value

$\frac{1}{K^{2}}$

is linearly proportional to the velocity V.

The processor 160 of the surgical system 100 (FIG. 1) normalizes thevalue

$\frac{1}{K^{2}\left( {x,y,t} \right)}$

and converts the normalized value into intensity of a pixel (x, y) ofthe laser speckle contrast image. Since

$\frac{1}{K^{2}\left( {x,y,t} \right)}$

is inversely proportional to the velocity V, if

$\frac{1}{K^{2}\left( {x,y,t} \right)}$

is small, the velocity is also small and intensity of the pixel (x, y)is low and, if

$\frac{1}{K^{2}\left( {x,y,t} \right)}$

is large, the velocity V is correspondingly large and the intensity ofthe pixel (x, y) is high. Thus, a portion of a vessel where the bloodflows slowly is illustrated darker than a portion of a vessel where theblood flows faster. However, the way of converting laser speckle intointensity is not limited by the equation presented above as the above isprovided as an example. Any correlation between the laser speckle andintensity can be made within the scope of this disclosure by a personhaving ordinary skill in this art.

Further, the intensities of pixels resulted from the LSCI processes maybe normalized, formatted for display, stored, and passed to otherprocesses such as noise reduction, pseudo color rendering, or fusionwith white light image, etc.

The optical image 700 of FIG. 7 may be generated by an optical lightsensor (e.g., the light sensor 420 of FIG. 4) prior to vessel sealingand prior to clamping of the tissue. The location 710 of FIG. 7corresponds to the location of the box 610 of FIG. 6. The optical image700 is readily usable by a clinician to see blood vessels in the wholeview of the image and to identify a location of the vessel at thelocation 710, while the box 610 of the raw laser speckle data 600 ofFIG. 6 does not show legible vessels. The optical image 700 may beoverlaid on an image from other modalities or displayed over anotherimage in a manner of augmented reality Based on the optical image 700,clinicians may seal the identified vessel prior to or concurrently witha surgical operation. The optical image 700 shows there are a bloodvessel and probably a blood flow at the location 710.

FIG. 8 shows an LSCI 800 after sealing the vessel. The location 810corresponds to the location of the box 610 of FIG. 6 and the location710 of FIG. 7. As shown, there is no blood flow at the location 810,meaning that the blood vessel at the location 810 has no blood flow andthus has been sealed completely. Thus, by comparing an LSCI, which hasbeen obtained prior to vessel sealing, with the LSCI 800, which has beenobtained after the vessel sealing, the surgical system may be able toconfirm that vessel sealing has been completed.

In embodiments, identification of the locations of vessels andconfirmation of vessel sealing may be performed based on Ramanspectroscopy, which enables rapidly capturing the molecular environmentof tissues without destroying or altering the tissues. FIG. 9illustrates an energy delivery device 900 for identifying vessel intissue based on Raman shifts. The energy delivery device 900 includesfirst and second jaw members 910 and 920 configured to grasp tissue 935.The second jaw member 920 includes a laser source 930 having at leastone frequency and the first jaw member 910 includes a light sensor 970.

The laser source 930 irradiates a laser light 960 onto the graspedtissue 935. The laser light 960 may include a laser having a frequencyof 375 nm and/or 405 nm. When the laser light 960 passes through thegrasped tissue 935, it is scattered. The light sensor 970 then detectsor senses the scattered light 965. The light sensor 970 may include amicroscope 975, a spectroscope 980, and a detector 985. The microscope975 may be able to extract information in a minute area less than orequal to 1 μm with a help of a filter. The spectroscope 980 mayincorporate an appropriate diffraction grating to obtain a correspondingspectral resolution so that the detector 985 may be able to detect aRaman shift.

Raman spectroscopy generates information-rich spectra that, whencombined with chemometrics, provide powerful insight into the moleculardiversity within tissue. For example, information regarding amino acids(e.g., amide bonds between amino acids and their tertiary structure) canbe extracted and analyzed based on the Raman spectroscopy.

When a laser light 960 is irradiated on tissue by the laser source 930,photons are absorbed and scattered by the tissue 935. The Raman effectarises when an energy incident to the tissue 935 is different from anenergy scattered by the tissue 935. Different constituents havedifferent Raman effects. With this difference, a photonic energy shiftoccurs in the scattered light 965. For example, when the incident energyis larger than the scattered energy, Stokes scatter occurs, and when theincident energy is smaller than the scattered energy, anti-Stokesscatter occurs. Since the energy shift is small, Raman shift iscalculated by subtracting a reciprocal of the wavelength scattered froma reciprocal of the wavelength incident, namely:

${\upsilon = {\frac{1}{\lambda_{incident}} - \frac{1}{\lambda_{scattered}}}},$

where υ is a Raman shift in wave number, λ_(incident) is a wavelength ofthe light incident to the tissue, and λ_(scattered) is a wavelength ofthe light scattered by the tissue. Thus, the wave number of the Ramanshift has a unit of

$\frac{1}{distance}$

or cm⁻¹.

For example, Raman bands corresponding to C—C stretch of proline (855cm⁻¹), C—C stretch of hydroxyproline (874 cm⁻¹), C—N stretch of proline(919 cm⁻¹), proline (1043 cm⁻¹), and Amide 3 (1245-1270 cm⁻¹) arenotable. The hydroxyproline and two proline peaks are specifically Ramancollagen assignments confirming a collagen presence. Non-collagen richtissue indicative of biological tissue includes bands corresponding tocholesterols (699 cm⁻¹), phenlalanine (1003 cm⁻¹), C—H deformation ofproteins (1262 cm⁻¹) and carbohydrates (1342 cm⁻¹), amide II (1480cm⁻¹), and amide I (1663 cm⁻¹).

For exemplary purposes only, spectra showing an abundance value greaterthan 0.6 of a collagen rich end-member were selected and the mean ofthese spectra was then calculated for each graph. These means were thencompared between healthy and sealed areas to identify changes in thecollagen environment due to sealing via a difference spectrum. Sealedporcine blood vessel tissue, for example, have shown changes in the1252-1261 cm⁻¹ peaks and a shift to lower wave-numbers in the 1447 cm⁻¹peak. The 1600-1650 cm⁻¹ Amide 1 band showed a shift to higherwave-numbers. For bowel tissues, only the samples which were sealed withno compression and at 0.2 mega pascal (MPa) compression were used forcomparison as these sample maps included more than 3 spectra which metthe threshold requirements. The changes in the collagen rich spectrabetween sealed and healthy areas were less pronounced in the porcinebowel tissue samples when compared to sealed blood vessels. Incomparison to sealed blood vessels, bowel tissue sealed at 0.2 MPacompression pressure demonstrated similar trends in the protein bandshifts, specifically in the three broad protein bands, 1245-1270, 1445,and 1665 cm⁻¹, corresponding to the Amide 3, CH₂ bending, and Amide 1bands, respectively, though less distinct. In bowel tissue sealedwithout compression, band shift trends included the 1245 and 1665 cm⁻¹Amide 3 and Amide 1 band, respectively; however, less dramatic shiftswere seen in other protein bands (FIGS. 10A-10C).

Referring to FIGS. 10A-10C, Raman spectra are shown of healthy,RF-sealed collagen rich tissue areas from RF sealed porcine blood vessel(FIG. 10A), RF-sealed porcine bowel tissue without compression (FIG.10B), and at 0.20 MPa compression pressure (FIG. 10C). The 1313 cm⁻¹,1324 cm⁻¹, 1252-1261 cm⁻¹ and 1600-1690 cm⁻¹ peaks are highlightedcorresponding to the CH₃CH₂ twisting and wagging mode of collagen,respectively; Amide 3 and Amide 1 (nonreducible collagen crosslinks atlower wavenumbers and reducible collagen crosslinks at higherwavenumbers) respectively.

For thermal denaturing of collagen, the 1660 cm⁻¹ band is shifted tohigher wave-numbers in the sealed tissue area, thereby suggesting anincrease in reducible crosslinks and a decrease of non-reducible crosslinks within the collagen. Additionally, a shift in the 1302 cm⁻¹ peakto higher wave-numbers has been identified in collagen thermaldenaturing. Changes in the 1313 cm⁻¹ and 1324 cm⁻¹ peaks signifyingchanges in the CH₃CH₂ twisting and wagging modes of collagen alsodemonstrated a disruption to the native collagen. Lastly, the apparentshift of the 1252-1261 cm⁻¹ peaks to lower frequencies also implicatecrosslinks may have been reduced or broken. RF sealing of the porcinebowel tissue demonstrated less pronounced differences; however, sealingperformed at 0.2 MPa compression pressure demonstrated many of the samechanges, including shifts in the 1252-1261, 1313, 1324, 1443, and 1660cm⁻¹ bands, seen in the sealed blood vessels, again indicating adenaturing of collagen and, more specifically, a decrease innon-reducible cross links and an increase in reducible cross links asseen in FIG. 10C.

This molecular restructuring appears to be less collagen dependent asshown in the Raman difference plots in FIG. 10B with collagen differencespectrum highlighting fewer distinct band shifts in compressed tissueversus non-compressed tissue. This may be attributed to more collagenbond restructuring with the additional mechanical pressure duringsealing introduced with tissue compression.

As described above, the blood vessel has its own characteristics infrequency shifts (i.e., Raman shifts) based on molecular bonds (e.g.,CH₂ bond, amino bond, etc.) and structure (e.g., CH₃CH₂ twist andwagging mode of collagen). When the blood vessel is sealed, itsmolecular bonds and structure are changed, thereby changingcharacteristics of the Raman shifts. Thus, based on frequency shifts inRaman spectroscopy, blood vessels may be identified and sealing of theblood vessels can be confirmed in vitro. Further, the Raman spectra mayidentify strong or weak seal by conducting Raman spectroscopy over thesealing area.

By finding a Raman shift, which is only identified in blood vessels, thesame can be identified and, by finding another Raman shift, which can befound only in sealed vessels, sealing of the vessels can be confirmed.

FIG. 11 shows a flowchart illustrating a method 1100 for identifyingblood vessels and confirming sealing of blood vessels based on opticallight. The method 1100 starts by providing to a surgical system a 3Dmodel of a patient in step 1105. The 3D model may include a region ofinterest in the patient based on CT image data or other image data inthe DICOM format. In an embodiment, a live video may be provided to showanatomical structures or the 3D model may be generated from a livevideo. The surgical system may include EM tracking sensors fixedly orslidably installed on an end effector having two jaw members.

In step 1110, the end effector grasps tissue of interest with two jawmembers. One jaw member may include an optical light source and theother jaw member may include a light sensor. In step 1115, the opticallight source illuminates an optical light to the tissue. When theoptical light passes through the tissue, the light is scattered and thelight sensor receives the light, which has passed through and beenscattered by the grasped tissue in step 1120. The light sensor may be aCCD or CMOS and generate an image based on the received light.

In step 1125, the received light is processed to identify bloodvessel(s). In an aspect, the generated image may be processed toidentify the blood vessel(s).

As described above, an EM tracking sensor is installed on one of the twojaw members such that the location of the optical light source or thelight sensor may be estimated by the EM tracking sensor. Further, inconsideration of the estimated location, the location of the identifiedblood vessel(s) may be identified. In step 1130, the location of theidentified blood vessel(s) may be synchronized with the 3D model. Inother words, the location of the identified blood vessel in the realworld is mapped to the 3D model of the patient, so that thecorresponding location of the identified blood vessel in the 3D modelcan be calculated.

In step 1135, the identified blood vessel(s) is displayed, as agraphical representation, at the calculated location with respect to the3D model on a display. Thus, the clinicians performing a surgicaloperation may be able to locate a surgical device at a proper positionbased on the synchronized location of the identified blood vessel(s) asdisplayed in the display.

After positioning the surgical device at the proper location, surgicalenergy may be supplied to the tissue via the surgical device to seal theidentified blood vessel(s) in step 1340. In an aspect, the surgicalenergy may be RF, microwave, or electromagnetic energy.

After sealing the tissue, proper sealing of the identified vessel(s) canbe confirmed. In this regard, the grasped tissue is illuminated again bythe optical light source in step 1145. Then, the light sensor againreceives the light, which has passed through and been scattered by thegrasped tissue in step 1150.

In step 1155, the received light is processed to identify vessel(s),which corresponds to the identified vessel in step 1125.

In step 1160, it is determined whether or not the identified vessel(s)has been sealed completely. As described above in step 1125, thereceived light may be used to generate an image. In this regard, priorto sealing, a first image may be generated and, after sealing, a secondimage may be generated. Based on image processing, a vessel may beidentified in the first and second images. The identified vessel in thefirst image is then compared with the corresponding vessel in the secondimage. In a case when the dimensions, optical properties, etc. (absoluteor relative to the previously identified vessel) of the currentlyidentified vessel indicate that the currently identified vessel has beensufficiently sealed, the sealing of the identified vessel is confirmed.

Image processing is not limited to the above-described ways but may beperformed in other ways, which are readily appreciated by a personhaving ordinary skill in the art, to confirm completeness of sealing ofthe identified vessel.

When it is not confirmed that the identified vessel has been completelysealed, the method 1100 goes to step 1140 to further perform sealing ofthe identified vessel. When the sealing is confirmed in step 1160, themethod 1100 is ended.

FIG. 12 shows a flowchart illustrating a method for identifying bloodvessel(s) based on Raman shifts. The method 1200 starts by providing toa surgical system a 3D model of a patient in step 1205. In anembodiment, a live video may be provided to show anatomical structuresor the 3D model may be generated from a live video. The 3D model mayinclude a region of interest in the patient based on CT image data orother image data in the DICOM format. The surgical system may include EMtracking sensors fixedly or slidably installed on one of two jaw membersof an end effector.

In step 1210, the end effector grasps tissue of interest with the twojaw members. A laser source is installed on one of the two jaw membersand a light sensor is installed on the other one of the two jaw members.In step 1215, the laser source irradiates a laser light, which mayinclude one or more frequencies, to the grasped tissue. When the laserlight is irradiated into the grasped tissue, the laser light isscattered while passing through the grasped tissue. The scattered laserlight is detected by the light sensor in step 1220.

The scattered laser light includes frequency shift data, which is Ramanshifts. When there is a difference between received energy and emittingenergy by the grasped tissue, such the difference is expressed infrequency shifts. Further, when molecular structure or composition ischanged, the frequency shift also changes. That is, the Raman shift, thereceived scattered laser is processed to identify blood vessel(s) instep 1225. Since the blood vessel(s) has a specific frequency shiftdifferent from the other tissue elements, the blood vessel(s) can beidentified by combining areas where the specific frequency shift isdetected.

As described above, an EM tracking sensor is installed either one of thetwo jaw members. As described in step 1230 of FIG. 12, the location ofthe identified blood vessel(s) may be synchronized with the 3D model instep 1230.

In step 1235, the location of the identified blood vessel(s) isdisplayed on a display with respect to the 3D model. Thus, theclinicians performing a surgical operation may be able to locate asurgical device at a proper position based on the synchronized locationof the identified blood vessel(s) as displayed in the display.

After positioning the surgical device at the proper location, surgicalenergy may be supplied to the tissue via the surgical device to seal theidentified blood vessel(s) in step 1240. In an aspect, the surgicalenergy may be RF, microwave, or electromagnetic energy.

After sealing the tissue, it is necessary to confirm that the identifiedvessel(s) has been sealed properly prior to or concurrently with furthertreatment to the tissue. In this regard, the laser source irradiates thelaser light again onto the grasped tissue in step 1245. Then, the lightsensor again detects the scattered light, which has passed through thegrasped tissue and has been scattered by the grasped tissue in step1250.

In step 1255, the detected scattered laser is processed to identifyvessel(s), which corresponds to the identified vessel(s) in step 1225.

In step 1260, it is determined whether or not the identified vessel(s)has been sealed completely. As described above in step 1225, thedetected laser may be processed to generate an image. In this regard,prior to sealing, a first image may be generated and, after sealing, asecond image may be generated. Based on image processing, a vessel maybe identified in the first and second images. The identified vessel inthe first image is then compared with the corresponding identifiedvessel in the second image. In a case when the dimensions, opticalproperties, etc. (absolute or relative to the previously identifiedvessel) of the currently identified vessel indicate that the currentlyidentified vessel has been sufficiently sealed, the sealing of theidentified vessel is confirmed.

Image processing is not limited to the above-described ways but may beperformed in other ways, which are readily appreciated by a personhaving ordinary skill in the art, to confirm completeness of sealing ofthe identified vessel.

When it is not confirmed that the identified vessel has been completelysealed, the method 1200 goes to step 1240 to further perform sealing ofthe identified vessel. When the sealing is confirmed in step 1260, themethod 1200 is ended.

FIG. 13 shows a flowchart illustrating a method 1300 for identifyingblood vessel(s) and confirming sealing of blood vessel(s) based on laserspeckle data. The method 1300 starts by providing to a surgical system a3D model of a patient in step 1305. In an embodiment, a live video maybe provided to show anatomical structures or the 3D model may begenerated from a live video. The 3D model may include a region ofinterest in the patient based on CT image data or other image data inthe DICOM format. The surgical system may include EM tracking sensorsfixedly or slidably installed on one of two jaw members of an endeffector.

In step 1310, the end effector grasps tissue of interest with the twojaw members. A laser source and a light sensor are installed on only oneof the two jaw members. In step 1315, the laser source irradiates alaser light, which includes only one frequency, to the grasped tissue.When the laser light is irradiated into the grasped tissue, the laserlight is scattered and reflected off from the surface of the graspedtissue. The scattered laser light is detected by the light sensor instep 1320.

The scattered laser light includes laser speckle data, which includesintensity fluctuation data. When there is a moving object in the areairradiated by the laser source, the intensity fluctuates according tothe movement of the moving object (e.g., circulating red blood cells)and thus forms a pattern different from the Gaussian distributionpattern. By analyzing the intensity fluctuation of these laser specklepatterns together with time, velocity of the moving object can beidentified. Based on the velocity of the red blood cells, a blood vesselmay be identified. In this way, the laser speckle data is processed toidentify blood vessel(s) based on the laser speckle data or patterns instep 1325.

As described in step 1230 of FIG. 12, the location of the identifiedblood vessel(s) may be synchronized with the 3D model in step 1330.

As described above, an EM tracking sensor is installed the one jawmember, where the laser light source and the light sensor are installed.In step 1335, the location of the identified blood vessel(s) isdisplayed on a display with respect to the 3D model. Thus, theclinicians performing a surgical operation may be able to locate asurgical device at a proper position based on the synchronized locationof the identified blood vessel as displayed in the display.

After positioning the surgical device at the proper location, surgicalenergy may be supplied to the tissue via the surgical device to seal theidentified blood vessel(s) in step 1340. In an aspect, the surgicalenergy may be RF, microwave, or electromagnetic energy.

After sealing the tissue, it is confirmed that the identified vessel hasbeen sealed properly prior to or concurrently with further treatment tothe tissue. In this regard, the laser source irradiates the laser lightagain onto the grasped tissue in step 1345. Then, the light sensor againdetects the light, as laser speckle data, which is scattered andreflected off from the surface of the grasped tissue in step 1350.

In step 1355, the received laser speckle data is processed to identifyvessel(s), which corresponds to the identified vessel(s) in step 1325.

In step 1360, it is determined whether or not the identified vessel(s)has been sealed completely. As described above in step 1325, thedetected laser speckle data may be processed to generate an imageincluding intensity corresponding to movements of the red blood cells.In this regard, prior to sealing, a first image may be generated and,after sealing, a second image may be generated. Based on imageprocessing, a vessel may be identified in the first and second images.The identified vessel in the first image is then compared with thecorresponding identified vessel in the second image. In a case when thedimensions, optical properties, etc. (absolute or relative to thepreviously identified vessel) of the currently identified vesselindicate that the currently identified vessel has been sufficientlysealed, the sealing of the identified vessel is confirmed.

Image processing is not limited to the above-described ways but may beperformed in other ways, which are readily appreciated by a personhaving ordinary skill in the art, to confirm completeness of sealing ofthe identified vessel. For example, ultrasound imaging modality may beused to detect a flow rate before, during, and after the sealing and toconfirm completion of the sealing.

When it is not confirmed that the identified vessel has been completelysealed, the method 1300 goes to step 1340 to further perform sealing ofthe identified vessel. When sealing is confirmed in step 1360, themethod 1300 is ended.

FIG. 14 is a block diagram for a computing device 1400 representative ofcombination of the processor 160, the display 170, the user interface165, and the memory 175 of FIG. 1 or the controller 245 of FIG. 2 inaccordance with embodiments of the present disclosure. The computingdevice 1400 may include, by way of non-limiting examples, servercomputers, desktop computers, laptop computers, notebook computers,sub-notebook computers, netbook computers, netpad computers, set-topcomputers, handheld computers, Internet appliances, mobile smartphones,tablet computers, personal digital assistants, video game consoles,embedded computers, and autonomous vehicles. Those of skill in the artwill recognize that many smartphones are suitable for use in the systemdescribed herein. Suitable tablet computers include those with booklet,slate, and convertible configurations, known to those of skill in theart.

In embodiments, the computing device 1400 includes an operating systemconfigured to perform executable instructions. The operating system is,for example, software, including programs and data, which manages thedevice's hardware and provides services for execution of applications.Those of skill in the art will recognize that suitable server operatingsystems include, by way of non-limiting examples, FreeBSD, OpenBSD,NetBSD®, Linux, Apple® Mac OS X Server®, Oracle® Solaris®, WindowsServer®, and Novell® NetWare®. Those of skill in the art will recognizethat suitable personal computer operating systems include, by way ofnon-limiting examples, Microsoft® Windows®, Apple® Mac OS X®, UNIX®, andUNIX-like operating systems such as GNU/Linux®. In embodiments, theoperating system is provided by cloud computing. Those of skill in theart will also recognize that suitable mobile smart phone operatingsystems include, by way of non-limiting examples, Nokia® Symbian® OS,Apple® iOS®, Research In Motion® BlackBerry OS®, Google® Android®,Microsoft® Windows Phone® OS, Microsoft® Windows Mobile® OS, Linux®, andPalm® WebOS®.

In embodiments, the computing device 1400 may include a storage 1410.The storage 1410 is one or more physical apparatus used to store data orprograms on a temporary or permanent basis. In embodiments, the storage1410 may be volatile memory and requires power to maintain storedinformation. In embodiments, the storage 1410 may be non-volatile memoryand retains stored information when the computing device 1400 is notpowered. In embodiments, the non-volatile memory includes flash memory.In embodiments, the non-volatile memory includes dynamic random-accessmemory (DRAM). In embodiments, the non-volatile memory includesferroelectric random-access memory (FRAM). In embodiments, thenon-volatile memory includes phase-change random access memory (PRAM).In embodiments, the storage 1410 includes, by way of non-limitingexamples, CD-ROMs, DVDs, flash memory devices, magnetic disk drives,magnetic tapes drives, optical disk drives, and cloud computing-basedstorage. In embodiments, the storage 1410 may be a combination ofdevices such as those disclosed herein.

The computing device 1400 further includes a processor 1430, anextension 1440, a display 1450, an input device 1460, and a network card1470. The processor 1430 is a brain to the computing device 1400. Theprocessor 1430 executes instructions which implement tasks or functionsof programs. When a user executes a program, the processor 1430 readsthe program stored in the storage 1410, loads the program on the RAM,and executes instructions prescribed by the program.

The processor 1430 may include a microprocessor, central processing unit(CPU), application specific integrated circuit (ASIC), arithmeticcoprocessor, graphic processor, or image processor, each of which iselectronic circuitry within a computer that carries out instructions ofa computer program by performing the basic arithmetic, logical, controland input/output (I/O) operations specified by the instructions.

In embodiments, the extension 1440 may include several ports, such asone or more universal serial buses (USBs), IEEE 1394 ports, parallelports, and/or expansion slots such as peripheral component interconnect(PCI) and PCI express (PCIe). The extension 1440 is not limited to thelist but may include other slots or ports that can be used forappropriate purposes. The extension 1440 may be used to install hardwareor add additional functionalities to a computer that may facilitate thepurposes of the computer. For example, a USB port can be used for addingadditional storage to the computer and/or an IEEE 1394 may be used forreceiving moving/still image data.

In embodiments, the display 1450 may be a cathode ray tube (CRT), aliquid crystal display (LCD), or light emitting diode (LED). Inembodiments, the display 1450 may be a thin film transistor liquidcrystal display (TFT-LCD). In embodiments, the display 1450 may be anorganic light emitting diode (OLED) display. In various embodiments, theOLED display is a passive-matrix OLED (PMOLED) or active-matrix OLED(AMOLED) display. In embodiments, the display 1450 may be a plasmadisplay. In embodiments, the display 1450 may be a video projector. Inembodiments, the display may be interactive (e.g., having a touch screenor a sensor such as a camera, a 3D sensor, etc.) that can detect userinteractions/gestures/responses and the like.

In still embodiments, the display 1450 is a combination of devices suchas those disclosed herein.

A user may input and/or modify data via the input device 1460 that mayinclude a keyboard, a mouse, or any other device with which the use mayinput data. The display 1450 displays data on a screen of the display1450. The display 1450 may be a touch screen so that the display 1450can be used as an input device.

The network card 1470 is used to communicate with other computingdevices, wirelessly or via a wired connection. Through the network card1470, the computing device 1400 may receive, modify, and/or update datafrom and to a managing server.

The embodiments disclosed herein are examples of the disclosure and maybe embodied in various forms. For instance, although certain embodimentsherein are described as separate embodiments, each of the embodimentsherein may be combined with one or more of the other embodiments herein.Specific structural and functional details disclosed herein are not tobe interpreted as limiting, but as a basis for the claims and as arepresentative basis for teaching one skilled in the art to variouslyemploy the present disclosure in virtually any appropriately detailedstructure. Like reference numerals may refer to similar or identicalelements throughout the description of the figures.

Any of the herein described methods, programs, algorithms or codes maybe converted to, or expressed in, a programming language or computerprogram. The terms “programming language” and “computer program,” asused herein, each include any language used to specify instructions to acomputer, and include (but is not limited to) the following languagesand their derivatives: Assembler, Basic, Batch files, BCPL, C, C+, C++,C#, Delphi, Fortran, Java, JavaScript, machine code, operating systemcommand languages, Pascal, Perl, PL1, scripting languages, Visual Basic,meta-languages which themselves specify programs, and all first, second,third, fourth, fifth, or further generation computer languages. Alsoincluded are database and other data schemas, and any othermeta-languages. No distinction is made between languages which areinterpreted, compiled, or use both compiled and interpreted approaches.No distinction is made between compiled and source versions of aprogram. Thus, reference to a program, where the programming languagecould exist in more than one state (such as source, compiled, object, orlinked) is a reference to any and all such states. Reference to aprogram may encompass the actual instructions and/or the intent of thoseinstructions.

It should be understood that various aspects disclosed herein may becombined in different combinations than the combinations specificallypresented in the description and accompanying drawings. It should alsobe understood that, depending on the example, certain acts or events ofany of the processes or methods described herein may be performed in adifferent sequence, may be added, merged, or left out altogether (e.g.,all described acts or events may not be necessary to carry out thetechniques). In addition, while certain aspects of this disclosure aredescribed as being performed by a single module or unit for purposes ofclarity, it should be understood that the techniques of this disclosuremay be performed by a combination of units or modules associated with,for example, a medical device.

What is claimed is:
 1. A method for identifying a vessel within tissue,the method comprising: providing a graphical representation of asurgical site; grasping the tissue of the patient with first and secondjaw members of an end effector including a location sensor; illuminatingoptical light into the grasped tissue at the first jaw member; receivingthe optical light that has passed through the grasped tissue, at thesecond jaw member; processing the received light to identify a vessel,which is encompassed within the grasped tissue; receiving locationinformation of the end effector from the location sensor; synchronizinga location of the identified vessel within the graphical representationbased on the location information; and displaying the identified vesselat the synchronized location in the graphical representation.
 2. Themethod of claim 1, further comprising: supplying energy to the tissuethrough the end effector to seal the identified vessel; illuminating theoptical light into the identified vessel after supplying the energy; andreceiving the optical light that has passed through the identifiedvessel.
 3. The method of claim 2, further comprising: processing thereceived optical light during or after supplying the energy.
 4. Themethod of claim 3, further comprising: confirming whether or not theidentified vessel is sealed.
 5. The method of claim 4, wherein theidentified vessel is confirmed to be sealed by comparing the identifiedvessel before supplying the energy with the identified vessel aftersupplying the energy.
 6. The method of claim 1, wherein the graphicalrepresentation is a 3D model or a video image.
 7. The method of claim 1,wherein the vessel is selected from the group consisting of a bilevessel, a lymph vessel, and a blood vessel.
 8. The method of claim 1,further comprising: generating an electromagnetic wave.
 9. The method ofclaim 8, further comprising: sensing the electromagnetic wave, whereinthe location information is based on the sensed electromagnetic wave.10. The method of claim 9, wherein the identified vessel is displayed inan augmented manner in the graphical representation.
 11. A system foridentifying a vessel within tissue, the system comprising: an endeffector configured to grasp tissue, the end effector including a firstjaw member and a second jaw member; an optical light source configuredto emit optical light to the grasped tissue and fixed at the first jawmember; a light sensor configured to receive the optical light that haspassed through the grasped tissue, and fixed at the second jaw member; alocation sensor configured to detect location information of the endeffector; a processor configured to receive a graphical representationof a surgical site, process the received light to identify a vessel,which is encompassed within the grasped tissue, and synchronize alocation of the identified vessel within the graphical representationbased on the location information; and a display configured to displaythe identified vessel at the synchronized location in the graphicalrepresentation.
 12. The system of claim 11, further comprising: anenergy supplier configured to supply an energy through the end effectorto seal the identified vessel, wherein the optical light source isfurther configured to illuminate the optical light into the identifiedvessel after supplying the energy, and wherein the location sensor isfurther configured to receive the optical light that has passed throughthe identified tissue.
 13. The system of claim 12, wherein the processoris further configured to process the received light during or aftersupplying the energy.
 14. The system of claim 13, wherein the processoris further configured to confirm whether or not the identified vessel issealed.
 15. The system of claim 14, wherein the processor confirms thatthe identified vessel is sealed by comparing the identified vesselbefore supplying the energy with the identified vessel after supplyingthe energy.
 16. The system of claim 14, wherein the graphicalrepresentation is a three-dimensional model or a video image.
 17. Thesystem of claim 11, wherein the vessel is selected from the groupconsisting of a bile vessel, a lymph vessel, and a blood vessel.
 18. Thesystem of claim 11, further comprising: an electromagnetic wavegenerator configured to generate an electromagnetic wave.
 19. The systemof claim 18, further comprising: an electromagnetic sensor configured tosense the electromagnetic wave, wherein the location information isbased on the sensed electromagnetic wave.
 20. The system of claim 11,wherein the identified vessel is displayed in an augmented manner in thegraphical representation.